A statistical mechanics approach for scale-free networks and finite-scale networks 
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We present a statistical mechanics approach for the description of complex networks. We first 
define an energy and an entropy associated to a degree distribution which have a geometrical inter- 
pretation. Next we evaluate the distribution which extremize the free energy of the network. We find 
two important limiting cases: a scale-free degree distribution and a finite-scale degree distribution. 
The size of the space of allowed simple networks given these distribution is evaluated in the large 
network limit. Results are compared with simulations of algorithms generating these networks. 
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Universality features of the structure of com- 
plex networks are very good candidate for a 
statistical mechanics treatment. While non- 
equilibrium aspects of these networks was first 
elucidated, later attention has been addressed to 
equilibrium properties of these graphs. In this 
paper we reveal some new property of finite-scale 
and scale-free networks. We characterize their 
different degree distribution as equilibrium de- 
gree distribution of a statistical mechanics prob- 
lem written in terms of an energy and an en- 
tropy having a graphical interpretation. We ob- 
serve that scale-free degree distribution for un- 
corrected networks and in the large size limit 
correspond to structure in which the space of dis- 
tinguishable simple graphs is strongly suppressed 
respect to the finite-scale distribution case. 

Complex networks describe the intertwin relations be- 
tween the elements of a variety of complex systems as dif- 
ferent as the Internet and the biological networks of the 
cell 0, 0, H[ . To understand the interplay between their 
functions and their structure it is natural to study how 
far from optimal performance are these networks. Conse- 
quently optimal networks have been defined respect to a 
specific function 0, H, i| , or ii) to their dynami cs B , H, B , 
or in) to some topological robustness features [10| . Nev- 
ertheless to understand the universal features of these 
networks which are found in many networks regardless 
of the specific dynamics which takes place of them, sta- 
tistical mechanics approaches are more appropriate. A 
major universal property of complex networks is their de- 
gree distribution P(k). In fact both scale-free networks 
P(k) ~ fc -7 with diverging second moment of the 
degree distribution (fc 2 ) and finite-scale networks with fi- 
nite second moment (fc 2 ) are widely encountered in many 
real networks. These two general network distributions 
have very strong impact also on the dynamics defined 
on these grap hs as it has been widely discussed in the 
literature jl2l . Il3j . 

Different statistical mechanics approaches have been 
used to characterize complex (scale-free and finite-scale) 
networks. In particular the attention has been first ad- 
dressed to the out-of-equilibrium dynamics which by the 
means of preferential attachment [ll[ generates scale- 



free degree distributions. This dynamics is justified 
by direct measurements for example in the case of 
the the World- Wide- Web [lj]. Secondly, different ap- 
proaches have been taken to assess if the scale-free de- 
gree distribution has to be considered as the equilib- 
rium distribution of some statistical mechanics problem 
[H, El EE EE El [M EL El, H, [H . If we restrict our- 
self to uncorrelated networks we see that the approaches 
can be classified into three major classes: the ones which 
see the emergence of a scale- free network structure at the 
phase transition [TH, EI ; the ones which assume a sort of 
preferential attachment in the dynamics [H EI the ones 
which consider some hidden variables associated to each 
node of the network s Models in the first two 

classes have strong relations. 

In particular in [21J an energy of the type E = 
— hi log(fci) was considered. The Montecarlo prob- 
ability for a single move at T — I involve a linear prefer- 
ential attachment mechanism with nodes of higher degree 
attracting links with higher probability. As a function of 
the temperature of the graph the model has two phase 
transitions. At the boundary between a sparse phase 
and a connected phase at a critical temperature T c ~ 1 a 
scale-free network with exponent 7 = 3 is found . In [l5j 
the authors define a minifield theory whose Feynman dia- 
grams are the networks of the ensemble. The networks of 
these ensemble map to a urn or "ball in the box" model 
[24j in which the "boxes" map to the nodes, the "balls" 
map to the edges of the graph and the probability of hav- 
ing fc edges at node i is given by p(k) oc fc~ 7+1 .The scale- 
free network ensemble is recovered in the case of trees 
or networks with finite number of cycles. The same ap- 
proach is extend further to graphs with cycles in [lol [l3 | , 
in which the authors show that the mapping to the " ball 
in the box" model regards a distribution p(k) oc k 1 . In 
the canonical ensemble of Ref. [20( , the authors describe 
an equilibrium network in which there is preferential at- 
tachment of nodes of high degree. The model again map 
to the urn model [2J] with the probability p(k) ~ fc 1 . 
Finally another urn ("ball in the box" model ) is pre- 
sented in [23| in which the same mapping between the 
network and the "ball/boxes" is made but the proba- 
bility p(k) — (fc!)' 3 is chosen. This choice of the p(k) 
probability involve some preferential attachment of the 
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dynamics when the heat-bath rule [25| is considered. A 
finite scale network if recovered when j3 = 1 for each node 
of the graph while a scale-free degree distribution with 
exponent 7 = 2 plus logarithmic corrections is obtained 
in the case of uniform distribution of effective (3i on each 
node i of the network. 

In the following paper we will present a statistical me- 
chanics treatment of complex networks in which we con- 
sider as thermodynamic quantities two terms having a 
precise graphical interpretation. The "energy" associ- 
ated with the degree distribution will the logarithm of the 
number of allowed equivalent simple networks is possible 
to draw given a degree sequence. The "entropy" associ- 
ated with a degree distribution will be the logarithm of 
the number of ways in which we can distribute L edges 
among the N nodes of the graph following the distri- 
bution {Nk}. From this statistical mechanics approach 
we observe the emergence either of scale-free networks 
or finite-scale networks without been close to a phase 
transition in the network. The models is very closed re- 
lated to the urn ("balls in the box") models but with urn 
("boxes") mapping to the degree of the nodes and not 
to the nodes themselves. On the same time the dynam- 
ical rules which give rise to the network directly derives 
from the choice of the energy function we make. The 
appearance of scale-free degree distribution in the op- 
posite limit of some finite-scale degree distributions can 
be put in relation with the findings of Ref. [26|, [27], [H| 
where in the contest of growing networks it was shown 
that the growing Cayley tree networks are described by 
opposite equations with respect to the growing scale-free 
networks. Nevertheless, here the network is not growing 
and the nodes are all equivalent since there is no fitness 
associated to the nodes. Although the dynamics mecha- 
nism giving rise to these distributions do not appear to be 
related to real phenomena occurring in evolving networks 
(as for example the preferential attachment mechanism) 
this approach has the advantage to characterize uncor- 
rected scale-free networks and finite scale networks on 
the base of the multiplicity of simple graphs is it possible 
to draw given a degree distribution. In fact we obtain 
that scale- free degree distributions characterize networks 
in which the energy is much higher than for finite-scale 
networks. In the paper we will show that this implies for 
uncorrelated simple graphs that the number of allowed 
distinguishable simple graphs given a certain degree dis- 
tribution is minimized for scale- free networks. This result 
indicates that in the swapping algorithm for randomiz- 
ing a network 29] the number of different graphs one 
can sample is actually reduced for scale free networks 
with low exponent 7. 

In order to define our partition function we consider a 
network with N nodes and 2L edges. We distribute the 
edges e = 1, . . . 2L, through the N nodes of the network 
and we assign to each edge a variable s e = 1, . . . , K in- 
dicating the degree of the assigned node. The degrees 
k takes the values k — 1, . . . , K with maximal allowed 
degree K. The degree distribution {Nj.} of a single as- 



signment {s e } is such that the total number of edges 
departing from nodes of degree k can be expressed as 
kNk = J2e=i $k,s e - We would like to study which are the 
equilibrium properties of the distribution of {s e } that 
maximize or minimize an energy functional E({s e }) . 
Therefore, considering a problem close to the original 
Backgammon model [30(, we define the partition func- 
tion 

1 j2'^ iE({Ss}) C 1 ) 



Z = 



(2L!) 



where the normalization factor (2L)! is chosen for con- 
venience and the sum is extended over all distributions 
{s e } with e = 1, . . . , 2L such that satisfy the conditions 



kN k = Y,S Se ,k 

e 

J2 N k = N. 



(2) 



In the following we will consider an the energy E({s e }) = 
E({Nk}) associated to the degree distribution of the net- 
work given by the logarithm of the number Ag of indis- 
tinguishable simple networks it is possible to draw given 
the degree sequence, i.e. 



E({N k }) = \og(M G ) 



(3) 



(where simple networks are networks without tadpoles 
and double links) . The number Ag can be expressed as 



(4) 



In fact, for every simple graph associated with the given 
distribution every permutation of the edges departing 
from each node generates the same set of links. These 
permutations are given by Ylk k\ Nk > an d consequently we 
derive Eq. For z > low energy configurations are 
weighted more while for z < high energy configurations 
are more weighted in the partition function Z defined in 
Eq. [T] Consequently we expect that in the system for 
z < we should bound the energy per particle and there- 
fore the maximal connectivity to a finite value K. Since 
the energy associated to the distribution {s p } only de- 
pends on the degree distribution {N^} we can write the 
partition function as a sum over the distributions {Nk} 
with given energy E({Nk}) and entropy S({N).}), i.e. 



Z 



1 



(2L)\ 



{N k } 



.±EUN k })+S({N k }) 



(5) 



The entropic term S({Nk}) is equal to the logarithm of 
the number of ways Af{N k } m which we can distribute 2L 
edges into any degree sequence {ki, . . . , fe/v} of distribu- 
tion {Nk}, or equivalently the number of sequences {s e } 
which correspond to a degree distribution {N^} and is 
given by 



{Nk} 



(2L)\ 



Uk(kN k 



(6) 
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For z > the equilibrium degree distribution {Nk} will 
minimize the free energy of the network F({N k }) — 
E({N k })-zS({N k }). For z < the equilibrium distribu- 
tion will maximize the free energy E({Nk}) — zS({N k }). 
The role of the parameter z is to measure a tradeoff be- 
tween the 'energetic' and the 'entropic' term in the def- 
inition of the free energy, as well as the temperature T 
in classical statistical mechanics. Respect to the other 
statistical mechanics problems described in the introduc- 
tion, this model can be cast into a "ball in the box" model 
with ball mapping to edges and "boxes" mapping to con- 
nectivity values and a weight p(k) — (fc!) _ ^/ fc associated 
to each edge of the graph linked to a node of degree k 
with p(k) oc k~P for large values of fc. In equation ([5|) 
the sum over the {N k } distributions is extended only 
to {Nk} for which the total number of nodes N and the 
total number of links L in the network is fixed, i.e. 
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To enforce these conditions we introduce in (JSJ) the delta 
functions in the integral form obtaining the expression 
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(2L)! 



dX f dv s—^ 
~oZ I ~oZ 2^ exp 
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FIG. 1: Average degree (k) of the distribution < N k . > as a 
function of v for networks with different value of z, starting 
with z — 0.2 (solid line at the bottom), and from bottom 
to top the curves with z = 0.3, 0.4, 0.5, 0.6, 0.7, 0.8 are drawn 
until the curve with z = 0.9 (dashed line at the top). 



and the marginal probability that Lk = kn = I is given 
by 

P(L k =l = nk) = Je^W^, (12) 



with 



-i\(2L - J2 kN k) ~ iv(N - J2 N k) 



Z k {L,i,N) 



dX 
2V 



dv 
2^ 



exp[JV/ fc (A,M)] (13) 



dX 
2^ 



dv 

2^ eXP 



-iX2L - ivN + 22 l °S G k (\ v) 

k 



where 

G ^-) = E(^yy-p{fciv fc 



and f k (X,v,£) = i((k) - £/N)X - iu(l - t/{kN)) + 
= 7r J2 s =ik ex P[«A + iv/a - log(s!)]. If we develop (JT2J) 
for I <c L and we use the Stirling approximation for fac- 
torials, we get that each variable Lk is a Poisson variable 
^ ' with mean < L k > satisfying 



< L k > 



--<N k >^k-^- 1 e x+ h^ k . 



(14) 



iX + iy -ilog(fc!) 

k zk 



(9) 

Assuming that the sum over all Nk can be approximated 
by the sum over all Lk = kNk — l,2,...oo we get 
logGfc(A, v) = exp [iX + iv/k — rr log(fc!)l and 

/(A, v) = -i(k)X - iv + 1 J]e tt ^*-AW i ( io) 



where < k >= 2L/N indicates the average degree of the 
network. By evaluating © at the saddle point, deriving 
the argument of the exponential respect to A and v 1 we 
obtain 



JX+iu/k- Jfc log(fe!) 



k 

N^k 

k 



(11) 



If we restrict ourself to the networks with finite average 
degree in the thermodynamic limit, the allowed values 
of z are z G (—1,1). From the expression (fT¥j) for < 
Nk >, if z G (0,1) the equilibrium degree distribution 
is scale-free with a power-law tail characterized by the 
exponent 7 = - + 1. The parameter v ^ modulates 
the average degree of the graph constituting for v > 
an effective lower cutoff of the distribution wherever the 
upper cutoff K of the degrees is the natural cutoff of 
the distribution (|14|l . A different scenario arises if z < 
0, when the equilibrium network, in average (|14|) has a 
power-law degree distribution increasing with the degree 
k. In this case the Lyapunov functions v and A cannot 
fix the average degree unless one introduces by hand an 
upper cutoff K in the degree of the nodes of the order of 
magnitude of the average degree (k). We note here that 
also for z G (0,1) it could be convenient to set by hand a 
structural cutoff K ~ N 1 / 2 for z > 1/2 in order to obtain 
an uncorrelated network. 
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FIG. 2: The distribution < N k > for 2 = -0.75 (panel 
(a)) and z = 0.75 (panel (b)) as a function of the av- 
erage degree of the network. Data are shown for (k) = 
2, 10, 20, 40, 100(dotted solid line) for z = -0.75 and for 
(k) = 1.3, 2„ (k) = 4 (solid line), (k) = 40, 100 for z = 0.75. 



The values of the Lagrangian multipliers A and v and 
of the upper cutoff K are fixed in the case z G (0, 1) by 
the conditions: 



v *e 
z 

(k) 



r(U)-r(^) 

z K z 



N 



' K 
1 1/ 



1 



.1 



1 f 



r y- r w 



= 1, 



(15) 



where T(a,b) indicates the incomplete Gamma function 
and (k) = 2L/N the average connectivity of the network. 
This system of equations is solvable provided (k) > 1, as 
can be seen from Figure [T] where we plot the average 
connectivity of a network as a function of v for different 
values of z. 

On the other hand, in the case z G (-1,0) we have to 
fix the upper cutoff K for the system to be stable and 
not crumpled. If we take K = (1 + z){k) the parameters 
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FIG. 3: Entropy of the network as a function of the parameter 
z, in the thermodynamic limit N — > 00 



v, A satisfy the following conditions, 

v = 

e x+ *K*z = N. 

(16) 

The two types of distributions described by (Q3J) with z > 
and the parameter of the distribution satisfying Eqs. 
(I15p , or with z < and the parameter of the distribution 
satisfying Eqs. (|16p . are significantly different. In fact, 
for z > there are highly connected nodes in the network 
and the degree distribution for large degrees k decays as 
a power-law with an exponent 7 fixed by the value of 
z. On the contrary, for z < the predicted distribution 
(P3J has a finite-scale and there are no highly connected 
nodes. Furthermore the topology of the network is very 
different from the one of the power-law case since the 
most connected nodes are also more abundant than less 
connected ones. In Figure [2] we show the distributions 
< Nk >which solve these equations for z positive and 
negative at different values of the average connectivity 
(fc) taking K = (1 + z){k) for z < 0. 

Given the distributions (fT^|) we can calculate the en- 
ergy of the network as a function of z at fixed aver- 
age connectivity (k), always fixing the upper cutoff to 
K = (1 + z)(k). In Figure[3]we present the energy of the 
network such that —< Nk > as a function of z for 
different average connectivities. 

The energy has a minimum in the limit z —> when the 
equilibrium degree distribution is such that < Nk > in 
infinitely peaked around the average connectivity. On the 
contrary, in the limit z — > 1 where the degree distribution 
has a power-law exponent 7 — > 2 the energy E({Nk}) is 
at the maximum. We note here that the energy E({Nk}) 
of complex networks calculated on the equilibrium degree 
distribution (|14[) . is an extensive quantity in both cases 
z > and z < 0. 

In the case z < and in the case z > where we 
introduce by hand a structural cutoff K ~ N 1 ^ 2 we can 
assume that the networks described in this paper are ran- 
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FIG. 4: MonteCarlo results for networks of N = 1000 nodes 
and L = 4000 links for different value of z. The Inset report 
the power-law exponent 7 of the distribution as a function of 
z (empty circles) and the predicted behavior (solid lines). 



domly wired. In this case we can evaluate the number 
Msg of distinguishable simple graphs it is possible to con- 
struct given the degree distribution (|14p . This number is 
approximated by 

1 ( <k 2 > \ 2 
(2L)!!e"v^^ 

K e *(W» 

In fact the total number of wiring it is possible to draw 
given 2L edges is given by (2L)\l. This number include 
all type of possible wiring of the edges including the ones 
which give rise to graphs which are not simple. Assuming 
that the graph is randomly wired, i.e. that the probabil- 
ity that a node with fc, edges connect to a node with kj 
edges is a Poisson variable with average kikj/{< k > N) 
the probability II that the graph is simple is equal to [31j 

n = TT (1 + hk J ) e -kik } <k>N „ e -h(^)\ 

11 V <k> N J 

(18) 

Finally in the expression (fTTj) for Msg there is an ad- 
ditional terms which takes into account the equivalent 
wiring of the edges which is given by e~ E ^ Nh ^K The 
term for scale-free graphs with cutoff K oc N 1 / 2 is 

subleading respect to the energetic term E{{Nk}) which 
dominates for large network sizes N. Consequently the 
number Msg of distinguishable simple graphs given a de- 
gree sequence is maximal for the graph with z = and 
minimal for the scale-free graph with 7^2. 



I. ALGORITHMS 

From the derivation of our model it is evident the 
connection with "ball in the box" problems [HI, [U, [3(| • 
In our approach the "boxes" map to the degree of the 
nodes and the "balls" map to the edges of the graph. 
This makes a crucial difference respect with the models 
[TEl [23j . in which the "boxes" map to the nodes of 
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FIG. 5: Degree distribution of the network evolving following 
algorithm II with different values of zeta. The networks have 
N = 500 nodes L = 1000 links have been evolved in 5000L 
timesteps for 10 runs and have a maximal cutoff K taken to 
beis:= [(k){l + z)]. 

the graph. The model, as well as a urn model can be 
simulated using a heat-bath rule [25J which then suggest 
the following algorithm: 

-Algorithm I- 

• Choose randomly a "ball" randomly, i.e. choose a 
link 

• Choose a "box" randomly, i.e. chose a node j' with 
a random value of its degree kj>. This choice is 
implemented by choosing a node j' with probability 
proportional to 1/Nf.(j>\. 

• Swap link with link with the heat bath 
rule algorithm, i.e. with probability II = (kji + 
1)-V* 

From the shape of the linking probability II we infer that 
this algorithm, include some sort of inverse preferential 
attachment since rewiring to nodes with lower degree are 
more frequents. In Figure [5] we report the resulting de- 
gree distributions as a function of z. The resulting dis- 
tribution differ from the theoretical prediction for the 
presence of sharp lower cutoff. Nevertheless the distribu- 
tion for large k decays with power-law exponents close to 
the expectation (see Inset Figure [4]) with the difference 
depending on finite size effects. We checked that the up- 
per cutoff K scale with the network size as K ~ N a and 
a (x z. 

For the case z < the algorithm follow a rule similar 
to the one used in the case z > with the further 
introduction of the ad hoc upper cutoff K which is 
needed for the stability of the algorithm. Consequently 
the algorithm is as following: 

-Algorithm II- 



6 



• Choose randomly a link 

• Choose node j' with a random value of its degree 
kji, i.e. choose a node j' with probability propor- 
tional to l/Nujiy 

• Swap link with link only if kji < K and 
in this case with probability II 



with 



n 



(kf + ir i/z 



(19) 



The results of the simulations following algorithm II are 
reported in Fig. [5] The algorithm requires a very long 
equilibration time respect to the case with z > 0. Conse- 
quently we consider relatively small networks sizes up to 
500 nodes. In the Inset we report the value the best value 
of the power-law fit to data together with the theoretical 
expectations. 



In conclusion, the statistical mechanics treatment of 
complex networks shown in this paper is able to put in a 
similar context, but in opposite limits, the emergence of 
scale-free networks and finite-scale networks. Scale-free 
degree distribution correspond to higher energy states of 
the network respect to finite-scale networks. Especially 
homogeneous random graphs obtained for z — have 
minimal energy. Heterogenous scale-free networks cor- 
respond to higher energy states. We have shown that 
this implies that uncorrelated simple scale-free networks 
live in a space of allowed networks relatively small. We 
believe that correlations present in various complex net- 
works describing many technological, biological and so- 
cial systems can only further reduce this space. Conse- 
quently the evolutionary rule by which this degree dis- 
tribution is selected reveal a tendency of minimizing this 
space. 
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